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Abstract 



Two well known directional tests of significance are presented within the 
multiple linear regression framework. Adjustments on the computed proba- 
bility level are indicated. The case for a directional interaction research 
hypothesis is defended. Conservative adjustments on the computed proba- 
bility level are offered and a more precise computation is requested of 
statisticians* Emphasis is placed more on the research question being 
asked than on blind adherence to conventional formulae. 

Introduction 
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The generalized F ratio within the context of multiple linear regression 
is known to be applicable to a large number of research questions. There 
is a class cf questions, though, which requires an adjustment in the 
probability level which is reported by canned computer programs. This 
reported probability level is for an equally divided "two- tailed” test 
of significance, but often the researcher has justified a "one-tailed 11 
te$t of significance. Indeed, whenever the research hypothesis contains 
directionality, then the required test of significance is "one-tailed." 

A good deal of the research hypotheses that appear in the literature 
develop a valid rationale for directionality but very few of them proceed 
to fully take advantage of their stated alpha level. One only needs to 
look at, for example, Volume 11 ,of the Journal of Personality and Social 
Psychology . Numerous articles in this issue propose directional hypotheses 
and proceed to use a non-directional test. Indeed, Levinger and Schneider 



(1969) Indicate 

O 




that the results lor one hypothseie was significant in 
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the direction opposite to that hypothesized. In reliability and validity 
research, the research hypothesis of necessity mist be directional. It 

is seldon that a researcher gets excited bout a negative reliability coeffi- 
cient. Likewise, the reseat cher hypothesis the s ign of the corre- 
lational value indicating validity. A negative rrelation would only 
be expected when two scales are measuring the sak. henomentn, hut one 
scale has been reversed. (In this case we would still have all of the cri- 
tical region in one tail of the sampling distribution.) 

There are at least three situations that might require a "one- tailed' 1 
test of significance: (l) a research hypothesis suggesting one treatment 

resulting in a higher mean than another treatment; (2) a research hypoth- 
esis specifying either a positive correlation between two variables or a 

negative correlation between two variables; and (3) a research hypothesis 
specifying a directional interaction. The first two situations are well docu- 
mented in the statistical literature, but the last is not mentioned. 

Case 1: Directional mean difference research hypothesis 

k T e must be careful in interpreting the probability associated with 
directional hypotheses because the full and restricted regression models 
are the same with a one-tailed test as with a two-tailed test. A non- 
directional research hypothesis would take the form: There is a difference 

in the mean effect of treatments and A directional research hypothesis 

would take the form: Treatment results in a larger mean effect than 

does treatment T 2 . The full model in both cases would be: 

Model lj Y^ ** a Q U + a^r^ *f ^12 + the model where: 

\ 

Y^ * the criterion vector. 

U * the unit vector. 

° a 1 if the Y^ score cornea from a person in treatment 1, 0 otherwise, 

T 2 =* a 1 if the Y^ score comes from a person in treatment 2, 0 otherwise. 
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a Q , a^, are weighting coefficients which will produce the 
smallest sum of squared components in the vector, 

E 1 is the error in prediction, or (Y -Y^), using the weighting 
coefficients and the predictor variables in the full model. 

For each of the above research hypotheses, the statistical hypothesis is: 
There is no difference in the (population) treatment means. The statistical 
hypothesis inplies the restriction: a^ * a^ . Forcing this restriction 
on the full model, we arrive at; 

Model 2; = a Q U + > the restricted model. 

All symbols are as defined before, with being the error in prediction using 
the weighting coefficients and predictor variables in the restricted model. 

The two models can of course be compared with the F test, and the 
associated probability value will be reported by most canned programs. 

The probability value is the probability of this large a discrepancy 
or, one larger occuring under the restriction that the two population means 
are equal. The first two rows in Table 1 indicate the state of affairs 
Wh^n fhe research hypothesis is non-dtrec tional . The reported probability 
value is for a non-direc tional test of significance and thus no correction 
is necessary. 

?' If we are concerned about differences in a given direction, then 
we must look at the sample means to see if the difference between the means 
is in the hypothesized direction. If the means are in the direction 
hypothesized, the third example in Table 1, then we must halve the reported 
probability level, for it indicates to the researcher how often he would 
expect this large a discrepancy in both directions. If the means are not 
ii) the hypothesized direction (the last example in Table 1), then we 
surely do not want to hold as tenable the research hypothesis. The correct 
probability level in this case is (1 - PROB),' where PROB is the reported 
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probability value. Since PROB can never be larger than 1, the smallest 
actual probability level can never be less than .50, i.e. can never lead 
to holding as tenable the research hypothesis. 

Pedagogically , one might want to illustrate the F distribution a 3 in 
Figure 1. The top half of the F distribution can be thought of as the 
F ratios resulting when Treatment 2 has a higher mean than Treatment 1. 

Thje bottom half then represents those F ratios resulting when Treatment 
1 has a higher mean than Treatment 2. It should be quite clear from 
Figure l that if one^ alpha level is .05 the appropriate lower limit 
for a non-direc tional test is F = 4.20, whereas if the research hypothesis 
involves directionality, then F = 2.89 is the appropriate lower limit 
(this being the tabled F value for alpha = 2 x .05, or for an alpha of .10; 
degrees of freedom equal l and 28). 

Case 2: Directional correlational research hypothesis 

The argument for this case is similar to the previous argument, the 
only difference is that here we have a continuous predictor variable rather 
than a dichotomous predictor variable. Often in correlational research, 
thfe research hypothesis is something like: There is a non-zero relationship 

between X^ and Y 2 * The statistical hypothesis in this case would be: 

There is a zero relationship between and Y 2 * The full and restricted 
mqdels would be; 

Model 3: Y 2 * a o u + a ]^i + ^ 3 * the model where; 

' ; , ,1 ’ Y 2 ■ the criterion vector* 

. '/ * ' ' t ; 

' . , . •*, * U ** the unit vector. 

X * the continuous predictor vector. 

V a Q and a^ are weighting coefficients which will provide the 

■ i sum of squared component- in the vector. 

Ej is the *error in prediction <Y 2 - Y 2 ) using the weighting 
* * - . coefficients and predictor variables in the full model. 

o' 'v; ’ 

restriction: a^ 0 0 results in 




Model 4: Y = a U + E^; the restricted model where all symbols are ^as 

above, and where Is the error in prediction (Y - Y^) 
using only the overall mean (Sq) . 

One 's research hypothesis might involve a directional relationship 
such as: There is a positive correlation between and Y^ . The full 

and restricted models would be the same, but again one would have to inspect 
the sign of the weighting coefficient to make sure the non-zero correlation 
is in the hypothesized direction. The same kinds of corrections in the prob- 
ability level are called for in this case as in the case for directional 
differences, and examples are depicted in Table 2. Indeed, we would 
expect this to be the case because the ter.t for the difference between two 
means is algebraically equivalent to the test of significance for the point 
biserial correlation, a special case of the Pearson Product Moment Correlation 
(|<elly, Beggs, McNeil, Eichelberger , and Lyon, 1969). 

Case 3: Directional interaction research hypothesis 

This third case has probably not been utilized in the literature 
because it has not been described in the standard statistical texts. 

We are not aware of any applied examples of this case, although many research 
hypotheses in the literature actually call for such an analysis. When 
a two-tailed interaction analysis is run on a directional interaction 
hypothesis rather than the legitimate one-tailed analysis, the researcher 
is reporting a probability level which is not indicative of the actual 

i; 

pobability. As in the previous cases, if the results are in the hypothesized 
direction the actual probability value is less than that which the researcher 
reports. We are not aware of the actual correction, as will be indicated shortly 

if ' 

An example from the literature may help clarify the problem, Gentile 
(1968) hypothesized? "the lower the sociocultural level of the student, 
the more he should benefit from the definition treatment (as compared to 
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the no-definition treatment )/ 1 Figure 2 illustrates the kind of interaction 
indicated by the research hypothesis. Figure 3 illustrates the other half 
of the situations wherein an interaction can occur. These kinds of inter- 
action in Figure 3 are evidently not of interest to Gentile. Therefore, 
the reported probability level should be at least halved if the results 
arp in the hypothesized direction. 

Ke say at least halved because there are other kinds of interactions 
similar to that depicted in Figure 1 which would not reflect the research 
hypothesis. Figure 4 contains one such situation wherein the definition treat- 
ment is inferior to the no-definition treatment. Again one would not want 
to hold as tenable the research hypothesis with this set of data. 

(. As in the first two cases, the full and restricted models for the 
directional and non-dirctiona! interaction questions are exactly the same 
(See Table 3). The sociocultural levels can be treated as categorical 
variables or as continuous, and \^e prefer the latter. (The discussion would 
become more involved if we didn't do it this way.) 

; The full model which allows interaction to occur would be: 



Tg ■ 1 if the subject received the no-definition treatment, 0 otherwise. 

X^ ■ sociocultral level of the subject if he received the definition 
. treatment, 0 otherwise. 

X 2 * sociocultural level of the subject if he received the no- 
definition treatment, 0 otherwise. 



MofJel 5 j * 3 » a Q U + a^ + a 2 T 2 + b X t + b 2 X 2 + E 5 



Vhere : 



Y 3 « the criterion vector. 
U * the unit vector. 








a Q , a2 , b , b2 are weighting coefficients which will produce 
the smallest sum of squared components in the vector. 

Eej = the error in prediction, (Y^* Y^) , using the wighting coefficients 
and predictor variables in the fuul model. 

In this example and b2 are the slopes of the straight lines of 

best fit for the two treatments. The hypothesis of no interaction in the 

population stipulates that the population slopes are equal (B^ = I^). 

Since the sample slopes are the best estimators of the population slopes, the 

restriction which does not allow interaction to occur is: b^b^. This 

restriction placed on the full model results in the following restricted 

mofiel; 

Model 6: Y3 - a Q U + a^ + a T 2 + b^ + E 6 

All symbols are as defined above, and where is the sociocultural level 
of the subject, no matter which treatment he received. E^ is the error in 



prediction, (Y3 - Y^) , using the weighting coefficients and predictor 
variables in the restricted model. Again, the full and restricted models 
can be compared via the generalized F ratio. 

If one has a non-d ireutional interaction question and the F is sig- 
nificant then the results can simply be plotted and the reported probability 
level reported. 

If one has a directional interaction question and the F is significant, 
then the results must be plotted to see if the interaction occurs in the 
direction hypothesized. If the results are opposite to that hypo thesized , 
we surely would not want to hold as tenable the research hypothesis, 

If the interaction is in the direction hypothesized, then the exact prob- 
ability is at least one-half the reported probability. 

Jfe feel that the above adjustment is not- an exact adjustment, but at 
this time we are not able to describe the exact probability. Wt would 
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encourage researchers to consider this question and in the future try to 
develop the exact probability. Certainly though, the interaction plot 
must reflect the research hypothesis before the researcher can reject the 
statistical hypothesis and hold as tenable the research hypothesis. 

What we question is the probability statement associated with the 
interaction test of statistical significance when the researcher has stated 
a directional interaction research question. 7he reader should be reminded 
that the statistical hypothesis when testing either interaction or directional 
interaction is: There is no interaction, or, the lines are parallel. There 

are many ways of obtaining interaction and only a small subset of these is 
of interest to the researcher who is interested in a directional interaction 
question . 

These thoughts seem to be important because many decisions are based 
on statistical grounds whfcch are being used incorrectly. Many research 
hypotheses involve a directional hypothesis. The researcher is hurting 
hipse^f when he uses a two-tailed test rather than a one-tailed test. If 
his results are in the hypothesized direction, the statistic may not fall 
in the critical region of the two-tailed test, whereas it might have fallen 
in the critical region of the one-tailed test, (Please remember to a?so 
reporp che amount of variance being accounted for in either case, as that 
lnjiex will probably communicate more than will the probability value,) 

What is even more disheartening is to see a researcher develop a beautiful 
directional hypothesis and then report that his data indicate significance 
In the '’opposite direction." He has used a two-tailed test of significance 
for the directional hypothesis and has found that the statistic falls in 
the critical region. A little thought would indicate that the researcher 
cannot hold as tenable his directional hypothesis under these conditions. 
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He should report what he found and urge future researchers to develop 
directional hypotheses to correspond with his data; it is ironical to 
report something as being significant which was completely opposite to that, 
which was expected. In essence, the rationale behind the directional 
hypothesis may be incorrect, but that cannot be determined on the initial 
data. 
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Several hypothetical examples for 
correlational hypotheses 
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Several hypothetical examples 
for interaction hypotheses 
(Full model takes the form of Model 5 and 
restricted model the form of Model 6.) 
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Figure l 



Exemplary F destribution (dfj = 1, df = 28) indicating 
F ratios resulting under the statistical hypothesis of equal 
population means, The area depicted by vertical lines repre- 
sents those F ratios resulting when, say, Treatment 1 has a 
u igher sample mean than Treatment 2. The area depicted by the 
horizontal lines represents those F ratios resulting when 
research hypothesis is directional, then the researcher must 
use the tabled F value for (2 x alpha). This process is analogous 
to adjusting the reported probability values as indicated in 
Tables 1, 2, and 3. 
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Figure 3 

Schematic diagram representing other interactions which 
could occur but were of no interest to Gentile (1968). 
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Figure 4 

Schematic diagram representing lines similar to Figure 2 
but with the definition treatment consistently inferior to 
the no-definition treatment. 
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